Hi, when creating a custom tap, if I need to have different records_jsonpath and http_headers in the client.py based on different streams, how can we do this? In the below example records_jsonpath can depend on the schemas file name and API endpoint can depend on the endpoint used in the stream:

class duoStream(RESTStream):

"""duo stream class."""

@property

def url_base(self) -> str:

"""Return the API URL root, configurable via tap settings."""

# TODO: hardcode a value here, or retrieve it from self.config

return f'https://{self.config.get("domain")}'

records_jsonpath = "$.response.can be different[*]"

# Set this value or override get_new_paginator.

next_page_token_jsonpath = "$.next_page"  # noqa: S105

@property

def http_headers(self) -> dict:

"""Return the http headers needed.

Returns:

A dictionary of HTTP headers.

"""

start_time, end_time = self.get_timeframe()

# Create params dictionary

params = {

'mintime': str(start_time),

'maxtime': str(end_time)

}

headers = self.sign("GET", self.config.get("domain"), "API endpoint", params, self.config.get("skey"), self.config.get("ikey"))

headers["Host"] = self.config.get("domain").encode('utf-8')

return headers

Hello, I implement a something like this for URL Parameters def get_url_params( if(self.name == "YOURTAPNAME"):

I suppose we can override these in the streams.py too? example class MyStream(RESTStream): # ... other stream properties and methods ... @property def records_jsonpath(self) -> str: """Return the JSONpath to the list of records in messages.""" # Customize records_jsonpath based on stream definition if self.stream_name == "your_stream_name": return "$.response.your_specific_path[*]" else: # Default records_jsonpath or any other custom logic return super().records_jsonpath

I have not tried that my self, but I do implement some conditional logic in a stream to control child stream context. class EmployeeChangesStream(PayComSourceStream): def get_child_context(self, record: dict, context: t.Optional[dict]) -> dict: if("eecode" in record):

I guess as long as we can ovveride properties and methods in the client.py in the streams.py the outcome should be same? I am new to meltano so not sure of the program flow. Correct me if I am wrong. Thanks

I would think but I am not sure, I started using meltano this week and am still learning aswell.

Same boat 😄